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DATA  ITEM  DESCRIPTIONS 


I .  INTRODUCTION 


« 


^During  the  past  year,  Air  Force  Special  Projects 
Production  Facility  (AFSPPF)  management  personnel  and, 
in  particular,  the  management  operations  group,  have 
undertaken  efforts  to  improve  contractor  reporting.  -in- 
April  1970,  Peat,  Marwick,  Mitchell  &  Co.  (PMM&Co.)  was 
retained  to  assist  in  this  effort.  PMM&Co.' s  study  was 
directed  at  helping  AFSPPF  improve  its  capability  for  man¬ 
aging  research  and  development  (R&D)  projects  by  improving 
its  requirements  for  reporting  contractor  performance  data. 
This  final  report  is  the  product  of  the  study  and  presents 
PMM&G€Hr-Ls  recommendations  for  improving  contractor  report¬ 
ing  of  performance  data  and  the  use  of  that  data  by  the 
facility. 


Study  Objectives 

The  PMM&Co .  study  was  directed  at  the  following 
objectives: 

.  identify  opportunities  for  improvement  of 
contractor  data  reporting  for  R&D  projects; 

.  recommend  specific  reporting  requirements  to 
capitalize  on  the  opportunities  identified; 
and 

.  identify  and  recommend  methods  for  handling 
the  data  that  will  result  from  the  implemen¬ 
tation  of  the  new  reporting  requirements. 

The  major  criterion  considered  by  PMM&Co.  in  making  specific 
recommendations  for  contractor  reporting  requirements  was 
what  data  is  needed  by  AFSPPF  contract  monitors  to  support 
management  decision-making.  Additional  criteria  that  served 
as  further  constraints  on  PMM&Co. 's  efforts  were  that  the 
recommendations  be  implementable  in  the  facility's  environ¬ 
ment  and  that  they  be  cost  effective  for  use  on  small  proj¬ 
ects,  where  the  addition  of  a  significant  administrative 
burden  would  be  undesirable. 


Implementation  of  improved  contractor  data  reporting 
requirements  should  benefit  the  facility  by  providing 
better  overall  management  of  the  R&D  effort  through  better 
and  more  complete  data  for  management  decision-making.  More 
specifically,  the  data  will: 

.  provide  more  visibility  into  the  contractor's 
operations  and  his  progress? 

.  tend  to  prevent  surprises  to  the  government 
by  the  contractor,  such  as  unexpected  over¬ 
runs  ;  and 

.  identify  problems  early  in  a  program  to  pro¬ 
vide  AFSPPF  management  adequate  time  for  cor¬ 
rective  action  and  a  greater  range  of  feasible 
alternatives  for  such  action. 

PMM&Co.  believes  that  the  objectives  stated  above  have 
been  met  and  if  the  recommendations  are  implemented  these 
benefits  will  be  obtained. 


Study  Scope 


The  scope  of  PMM&Co. 's  investigations  under  this  con¬ 
tract  involved  the  R&D  activities  of  AFSPPF,  and,  in  particu 
lar,  the  facilities  management  of  R&D  contracts.  The  study 
involved  review  of  and  recommendations  concerning  contractor 
cost,  schedule,  and  technical  performance  data.  Further, 
efforts  were  directed  at  examining  how  the  acquired  data 
is  used  to  support  the  facility's  management  decisions  in 
the  R&D  area. 


Study  Approach 

The  study  was  performed  in  approximately  four  months; 
i.e.,  April  to  August  of  1970.  PMM&Co.  worked  in  conjunc¬ 
tion  with  AFSPPF  personnel  and,  in  particular,  with  AFSPPF 
management  operations  personnel.  The  methodology  followed 
to  develop  the  recommendations  included  in  this  report  is 
described  in  the  following  steps: 

.  AFSPPF  R&D  contract  files  were  reviewed  to 
determine  the  size  and  nature  of  the  R&D 
projects. 
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.  Existing  contractor  reporting  requirements 
and  actual  contractor  reports  were  reviewed. 

.  The  contract  monitors  were  interviewed  to 
determine  their  needs. 

.  Tentative  recommendations  were  developed 
by  PMM&Co.  based  on  the  steps  above.  These 
recommendations  were  supplemented  by  sample 
report  formats  and  sample  Data  Item  Descrip¬ 
tions  (DID) . 

.  The  recommendations  and  samples  were  dis¬ 
cussed  with  the  contract  monitors  and 
AFSPPF  management  personnel. 

.  The  results  of  these  interviews  were  con¬ 
sidered  in  preparing  the  final  recommendations. 

PMM&Co.  believes  that  the  recommendations  contained  in  this 
report  were  received  with  a  high  degree  of  approval  and  en¬ 
thusiasm  by  the  contract  monitors  and  AFSPPF  management 
personnel.  The  remaining  sections  of  this  report  delineate, 
in  detail,  the  results  of  PMM&Co. 's  study  for  AFSPPF. 


II.  SUMMARY  OF  FINDINGS 


This  section  summarizes  PMM&Co.'s  identification  of 
the  status  of  contractor  data  reporting  and  the  use  of 
that  data  as  it  existed  at  AFSPPF  during  this  study. 


Cost  and  Schedule  Reporting  Requirements 

Contractors,  in  many  instances ,  are  not  presently 
submitting  detailed  plans  of  their  efforts  to  AFSPPF  for 
review  and  approval.  Further,  those  contractor  plans  that 
are  used  are  frequently  changed  without  clear  identifica¬ 
tion  of  the  reason  for  or  magnitude  of  the  change.  Hence, 
there  is  often  no  framework  within  which  to  evaluate  con¬ 
tractor  performance  data. 

At  the  present  time,  no  formalized  methods  of  relat¬ 
ing  the  value  of  work  performed  by  a  contractor  to  the 
actual  costs  to  date  of  that  work  exist. 

Existing  Air  Force  data  reporting  formats  are  not 
closely  suited  to  AFSPPF  data  requirements. 


Technical  Performance  Reporting  Requirements 

Technical  performance  management  of  contractor  efforts 
is  performed  by  AFSPPF  personnel  on  a  personal  interaction 
basis.  However,  this  interaction  is  limited  by  the  monitor's 
workload,  the  technical  complexity  of  the  efforts  involved, 
and  the  geographic  dispersion  of  the  facility's  contractors. 


Management  Information  System  Requirements 

At  the  present  time,  contractor  performance  data  is 
communicated  reasonably  well  v/ithin  AFSPPF.  Hence,  there 
is  significant  question  within  the  facility  about  the  use¬ 
fulness  of  a  formalized  management  information  center. 
However,  a  more  formalized  approach  to  communicating  con¬ 
tractual  data,  emphasizing  the  technical  objective  to  be 
met,  was  felt  to  be  needed  by  AFSPPF. 
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III.  SUMMARY  OF  RECOMMENDATIONS 


« 


This  section  summarizes  the  PMM&Co .  recommendations 
concerning  AFSPPF  R&D  management  that  are  detailed  in  the 
following  sections  of  this  report. 


Cost  and  Schedule  Reporting  Recommendations 

1.  Contractors  should  be  required  to  submit  cost  and 
schedule  plans  at  the  beginning  of  a  contract  and  then  re¬ 
port  against  these  plans.  Specifically: 

.  The  plans  should  be  submitted  to  AFSPPF 
subsequent  to  contract  negotiations  for 
review  and  approval. 

.  Periodic  reports  should  be  reviewed  against 
the  plans  for  identification  and  analysis 
of  variances,  and  for  determination  of 
possible  management  action. 

.  Changes  to  the  plans  should  be  made  only 
when  the  contract  is  changed.  The  changes 
should  be  reviewed  and  approved  by  AFSPPF 
personnel. 

2.  AFSPPF  should  levy  reporting  requirements  on  their 
contractors  that  will  enable  them  to  determine  a  measure  of 
"earned  value."  Specifically: 

.  Earned  value  should  be  used  as  a  way  of 
determining  the  value  of  work  accomplished 
for  comparison  with  the  actual  cost  of  the 
work . 

.  Two  approaches  to  measuring  earned  value 
can  be  used: 

,  task  valuation  (TV) ;  and 
.  milestone  costing  (MC) . 


-5 


.  With  the  exception  of  fixed  price  con¬ 
tracts,  TV  should  be  used  on  most  AFSPPF 
contracts  in  the  R&D  area.  Milestone  cost¬ 
ing  will  be  generally  less  applicable  than 
TV  but  will  still  be  useful  on  a  number  of 
AFSPPF' s  smaller  cost-reimbursable  contracts. 

3.  A  set  of  "additive"  Data  Item  Descriptions  should 
be  used  to  implement  cost  and  schedule  reporting  require¬ 
ments  on  AFSPPF  contracts.  These  are: 

.  R&D  Contract  Status  Report/Narrative; 

.  Program  Schedule/Milestone  Accomplishment; 

.  Cumulative  Cost  Projection/Report; 

.  Milestone  Costing  Plan/Report; 

.  Task  Valuation  Plan/Report. 

The  R&D  Contract  Status  Report/Narrative  would  be  used  on 
almost  all  contracts  in  the  R&D  area.  The  subsequent  re¬ 
ports  would  be  used  for  the  larger,  more  complex  contracts 
and  would  build  upon  the  narrative  report.  For  example, 
the  Task  Valuation  Plan/Report  would  incorporate  the  re¬ 
quirements  of  the  R&D  Contract  Status  Report/Narrative,  the 
Program  Schedule/Milestone  Accomplishment,  and  the  Cumula¬ 
tive  Cost  Projection/Report. 

4.  At  the  present  time,  the  use  of  critical  path 
method  (CPM)  scheduling  and  reporting  of  more  than  one 
level  of  detail  on  AFSPPF  contracts  is  not  appropriate. 

Use  of  CPM  scheduling  will  be  appropriate  for  certain 

large  hardware  development  contracts  planned  by  the  facility. 


Technical  Performance  Reporting  Recommendations 

AFSPPF  contractors  should  be  required  to  use  Technical 
Achievement  Plans  (TAP)  on  major  hardware  development  proj¬ 
ects.  Specifically,  these  plans: 
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.  will  include  a  detailed  listing  of  equip¬ 
ment  performance  specifications  and  interim 
points  at  which  progress  toward  the  accom¬ 
plishment  of  these  specifications  can  be 
measured. 

.  should  be  approved  by  AFSPPF  personnel  at 
the  beginning  of  contractual  effort. 

2.  Technical  performance  reporting  against  the  tech¬ 
nical  achievement  plans  should  be  required. 

3.  Contractors  should  be  required  to  perform  inte¬ 
grated  variance  analysis;  i.e. ,  variance  analysis  that 
looks  at  cost,  schedule,  and  technical  performance  variances 
to  identify  root  problems. 

4.  Technical  achievement  plans  should  be  used  on 
study  contracts  with  some  modification.  In  particular, 
study  objectives  will  be  used  instead  of  equipment  speci¬ 
fications,  and  technical  achievement  events  will  be  review 
oriented  instead  of  test  oriented. 


Management  Information  System  Recommendations 

1.  The  contract  monitor's  full  responsibility  for 
meeting  the  cost,  schedule,  and  technical  objectives  of  a 
contractual  effort  should  be  reaffirmed.  This  should  in¬ 
clude  changing  the  designation  "contract  monitor"  to  "proj¬ 
ect  engineer"  or  "project  manager." 

2.  The  use  of  work  plans  by  contract  monitors  on  con¬ 
tracts  or  technical  efforts  should  be  required. 

3.  A  Management  Information  Notebook  (MIN)  should  be 
used  as  a  practical  alternative  to  a  management  information 

center.  Specifically:  < 

1 

.  the  creation  of  a  formal  Management  Informa-  \ 

tion  Center  is  not  recommended  due  to  its 
inflexibility  and  the  effort  required  to 
maintain  it. 
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.  the  MIN  will: 


.  include  information  from  work  plans 
on  each  contract; 

.  provide  th;  Commander  with  the  original 
planning  baseline  and  the  current  status 
of  each  contract;  and 

.  be  easily  maintained  and  flexible. 

4.  When  a  number  of  AFSfPF  contracts  lead  to  the  same 
technical  objective,  planning  should  be  performed  on  a*  tech¬ 
nical  area  basis. 
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IV.  COST  AND  SCHEDULE  REPORTING  RECOMMENDATIONS 


This  section,  addressed  to  Paragraph  4.1  of  the  con¬ 
tract  statement  of  work  anti tied  "Develop  Contractor  Cost 
and  Schedule  Reporting  Requirements,"  presents  recommen¬ 
dations  for  cost  and  schedule  reporting  requirements  to 
be  met  by  AFSPPF  contractors. 


Statement  of  the  Problem 

AFSPPF  has  been  faced  with  a  number  of  cost  overruns 
schedule  delays,  and  technical  problems  on  R&D  contracts 
that  have  often  been  unexpected  and  of  significant  magni¬ 
tude.  Since  these  problems  have  been  "surprises,"  the  op¬ 
portunity  for  AFSPPF  personnel  to  take  corrective  action 
has  been  limited.  The  inability  to  take  corrective  ac¬ 
tion,  in  conjunction  with  the  magnitude  of  the  problems, 
has  made  it  difficult  for  the  facility  to  effectively 
manage  a  number  of  R&D  efforts. 

One  approach  to  alleviate  these  problems  is  to  re¬ 
quire  periodic  (e.g.,  monthly)  cost  and  schedule  report¬ 
ing  from  contractors,  which  will  make  contractual  perform¬ 
ance  visible  to  the  government  and  provide  adequate  data 
for  decision-making.  However,  AFSPPF  personnel  have 
stated  that  they  sometimes  lack  adequate  data  to  evaluate 
(1)  contractor  performance  on  R&D  efforts,  and  (2)  the 
success  of  the  facility's  R&D  efforts  in  a  specific  tech¬ 
nical  domain.  In  particular,  data  has  often  not  been 
available  to  support  sucii  decisions  concerning  the: 

.  investment  of  additional  funds  in  a  con¬ 
tract  encountering  technical  or  cost 
difficulties; 

.  extension  of  contract  completion  dates; 

.  continuation  of  investing  funds  in  ex¬ 
ploration  of  a  given  technical  domain; 

.  evaluation  of  contractor  performance,  in¬ 
cluding  technical  efforts  and  management 
capabilities;  and 
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.  reorientation  of  a  contractual  effort. 


Objective 

PMM&Co.'s  objective  was  to  identify  and  recommend 
contractor  cost  and  schedule  reporting  requirements  which 
would  provide  data  to  support  the  decisions  outlined 
above.  To  a  significant  degree,  fulfilling  this  objec¬ 
tive  involved  building  upon  the  reporting  requirements 
that  the  facility  has  been  implementing  in  the  past  year. 
Existing,  altered,  or  new  requirements  that  were  con¬ 
sidered  were  recommended  on  the  basis  of  their  contribu¬ 
tion  toward  meeting  the  following  specific  objectives  of 
reporting  requirements: 

.  to  provide  early  identification  of  defi¬ 
ciencies  in  contractor  performance; 

.  to  be  relevant  to  AFSPPF  management 
decisions; 

.  to  provide  visibility  into  contractor 
performance  (i.e.,  a  periodic  report 
should  indicate  a  problem  area,  but  it 
will  not  necessarily  provide  axl  the  data 
about  the  problem) ; 

.  to  require  a  minimum  of  contractor  prepa¬ 
ration  cost  and  AFSPPF  review  time;  and 

.  to  match  the  detail  of  reporting  with  con¬ 
tract  dollar  size,  type  (e.g.,  cost  reim¬ 
bursable,  fixed  price) ,  technical  risk, 
and  criticality  to  AFSPPF  requirements. 


Approach 

The  four  elements  of  contractor  cost  and  schedule  re 
porting  considered  by  PMM&Co.  for  potential  use  on  AFSPPF 
contracts  were: 

.  development  of  planning  baselines; 


-10- 


« 


.  development  of  a  measure  of  "earned  value"; 

.development  of  the  reporting  recommenda¬ 
tions  in  a  manner  consistent  with  Air  Force 
Data  Management  requirements  as  described 
in  AFSC/AFLC  Manual  310-1,  "Management  of 
Contractor  Data  and  Reports";  and 

.  use  of  simplified  critical  path  methodolo¬ 
gies  (CPM)  to  develop  schedule  data,  and 
the  reporting  of  more  than  one  level  of 
detail  for  large  contracts. 

Development  of  Planning  Baselines 

The  usefulness  of  contractor  periodic  reports  is 
often  limited  because  no  baseline  plan  exists  against 
which  to  evaluate  the  reported  performance.  The  fact 
that  a  particular  milestone  was  completed  during  a  period 
has  little  significance  unless  the  government  contract 
monitor  knows  when  the  milestone  was  supposed  to  be  com¬ 
pleted.  Hence,  PMM&Co. 's  efforts  to  develop  reporting 
requirements  considered  the  need  for  baseline  plans  to  be 
reported  against. 

Development  of  a  Measure  of  "Earned  Value" 

Earned  value  represents  a  way  of  associating  the 
value  of  work  accomplished  with  the  actual  costs  expended 
on  that  work  to  date.  Earned  value  is  defined  as  the 
originally  planned  value  (or  budgeted  cost)  of  the  work 
that  has  been  accomplished,  and  is  measured  independently 
from  the  actual  cost  of  performing  the  work.  The  actual 
cost  of  the  work  is  then  compared  with  its  budgeted  cost 
(or  earned  value)  to  determine  whether  an  overrun  or 
underrun  occurred.  Earned  value  represents  one  way  of 
separating  the  cost  and  schedule  components  of  a  spending 
variance. 

The  need  for  the  earned  value  approach  is  demon¬ 
strated  by  the  situation  presented  in  Figure  1. 
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Figure  1 


CONTRACTOR  COST  REPORT 


Cumulative 

Cost 


The  figure  shows  actual  costs  running  below  planned  costs, 
which  implies  that  the  contract  is  underrunning.  However, 
this  situation  could  be  caused  by  schedule  slippages  and, 
hence,  not  represent  a  true  underrun.  In  fact,  the  ac¬ 
tual  cost  curve  could  be  above  the  projected  curve  and 
not  indicate  a  problem  if  the  project  was  ahead  of 
schedule.  Since  earned  value  relates  only  to  the  value 
of  the  work  completed  independent  of  its  scheduled  or  ac¬ 
tual  completion  date,  spending  variances  because  of  sched¬ 
ule  variances  are  removed  from  the  cost  variance. 

Development  of  the  Reporting  Recommendations  in  a 
Manner  Consistent  With  Air  Force  Data  Management 
Requirements  as  Described  in  AFSC/AFLC  Manual  310-1 

This  manual  contains  the  Air  Force  policies  and  pro¬ 
cedures  relating  to  data  management.  As  such,  it  gives 
detailed  guidance  on  the  use  of  DD  Form  1664,  "Data  Item 
Description,"  and  DD  Form  1423,  "Contract  Data  Requirements 
List."  The  recommendations  made  in  PMM&Co.'s  report  are 
consistent  with  the  requirements  of  AFSC/AFLC  Manual  310-1. 
Where  specific  actions  are  necessary  for  AFSPPF  to  meet 
the  manual's  requirements,  such  as  gaining  approval  of 
certain  Data  Item  Descriptions,  they  will  be  described. 
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Use  of  Simplified  CPM  To  Develop  Schedule  Data, 
and  the  Reporting  of  More  Than  One  Level  of 
Detail  for  Large  Contracts 


On  large  Department  of  Defense  R&D  contracts,  PERT 
CPM  and  work  breakdown  structures  (WBS)  are  often  used  as 
management  tools.  PERT  CPM  is  used  as  a  scheduling  tool 
to  project  the  tasks  required  for  contract  completion  and 
to  relate  the  interdependencies  between  these  tasks.  A 
work  breakdown  structure  divides  a  large  project  into 
smaller  packages.  Work  planning,  reporting,  and  control 
are  then  related  to  these  smaller  packages. 


Recommendations 


The  results  of  PMM&Co.'s  efforts  to  develop  cost  and 
schedule  reporting  requirements  for  AFSPPF  contracts  have 
indicated  that  (1)  the  facility  should  require  contrac¬ 
tors  to  submit  cost  and  schedule  plans  and  report  against 
these  plans;  (2)  there  are  two  effective  ways  of  develop¬ 
ing  a  measure  of  earned  value  on  facility  contracts;  (3) 
the  required  reporting  data  can  be  acquired  by  the  fa¬ 
cility  using  a  set  of  reporting  formats  that  meet  the  re¬ 
quirements  of  AFSC/AFLC  Manual  310-1;  and  (4)  the  fa¬ 
cility  does  not,  at  this  time,  have  contracts  for  which 
CPM  or  a  WBS  are  necessary. 

Implementation  of  the  recommendations  will  be  accom¬ 
plished  by  using  five  Data  Item  Descriptions  (DID)  that 
PMM&Co.  has  written  and  recommended  for  the  facility's 
use.  Hence,  the  recommendations  discuss  the  following 
DID's  in  detail. 

I.  R&D  Contract  Status  Report/Narrative 

II.  Program  Schedule/Milestone  Accomplishment 

III.  Cumulative  Cost  Projection/Report 

IV.  Milestone  Costing  Plan/Report 

V.  Task  Valuation  Plan/Report 

These  DID's,  with  sample  reports,  as  appropriate,  are 
written  to  their  specifications  and  provided  in 
Appendix  A. 
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The  following  pages  contain  detailed  discussions  of 
PMM&Co.'s  cost  and  schedule  reporting  recommendations. 

Contractors  should  be  required  to  submit  cost  and 

schedule  plans  at  the  beginning  of  a  contract  and 

then  report  against  these  plans. 

Plans  should  be  submitted  to  AFSPPF  subsequent  to 
contract  negotiations  for  review  and  approval.  The  peri¬ 
odic  reports  should  be  reviewed  against  the  plans  for 
identification  and  analysis  of  variances  and  for  determi¬ 
nation  of  possible  management  action.  Changes  to  the 
plans  should  be  made  only  wnen  the  contract  is  changed, 
and  any  changes  made  at  this  time  should  be  reviewed  and 
approved  by  AFSPPF  personnel. 

The  facility,  on  present  contracts,  usually  does  not 
receive  a  cost  or  schedule  plan  from  contractors  at  the 
beginning  of  an  effort.  The  plans  that  are  available  may 
have  been  included  in  the  contractor's  proposal  or  are 
contained,  by  implication,  in  the  contractor's  first  pe¬ 
riodic  report.  However,  plans  from  contractor  proposals 
are  not  submitted  in  response  to  a  specific  facility  re¬ 
quirement  and  are  usually  modified  by  contract  negotia¬ 
tions.  Periodic  reports  usually  do  not  contain  detailed 
buildup  data  and  are  received  well  after  the  start  of  the 
contract.  Hence,  neither  of  these  plans  provides  a  de¬ 
tailed  baseline  against  which  variances  can  be  measured 
and  performance  evaluated. 

The  DID's  recommended  for  AFSPPF  use  provide  for  the 
submission  of  both  plans  and  reports  by  AFSPPF  contrac¬ 
tors.  The  first  submission  will  be  a  plan,  and  subse¬ 
quent  submissions  will  be  reports  against  that  plan.  For 
example,  the  first  submission  by  the  contractor  for  the 
Program  Schedule/Milestone  Accomplishment  data  item  would 
actually  be  the  program  schedule,  expressed  in  milestones. 
Successive  reports  would  then  show  program  progress  in 
the  form  of  milestone  accomplishment. 

The  contractor  should  not  change  the  original  plan 
unless  a  contract  modification  is  made.  As  the  effort 
progresses,  some  rescheduling  and  reestimating  will  occur. 
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Whenever  this  happens,  the  original  planning  data  should 
always  be  shown.  Otherwise,  changes  are  often  not  noted 
and  the  new  plan  loses  its  historical  context.  Further, 
if  changes  occur  in  cost  buildup  data,  such  as  overhead 
rates,  these  should  also  be  reported. 

AFSPPF  should  levy  reporting  requirements  on  their 
contractors  that  will  enable  them  to  determine  a 
measure  of  "earned  value." 


Two  ways  of  obtaining  an  effective  measure  of  earned 
value  of  AFSPPF  contracts  have  been  developed  and  are 
recommended  for  facility  use.  These  have  been  designated 
Milestone  Costing  (MC)  and  Task  Valuation  (TV)  and  are 
provided  as  DID  Forms  IV  and  V.  Sample  MC  and  TV  reports 
written  to  the  specifications  of  the  DID's  are  presented 
in  Appendix  A. 

TV  includes  a  program  schedule  showing  tasks  as  a 
function  of  time  and  a  subdivision  of  cost  by  task. 

These  elements  are  used  in  the  same  manner  as  in  the  Pro¬ 
gram  Schedule/Milestone  Accomplishment  and  Cumulative 
Cost  Projection/Report.  However,  TV  relates  the  cost  and 
schedule  data  by  placing  a  value  (in  dollars)  on  each 
task.  Each  reporting  period,  an  estimate  of  the  percent 
complete  of  each  task  is  made.  The  percent  complete  is 
then  multiplied  by  the  estimated  value  of  the  task,  giv¬ 
ing  a  measure  of  earned  value  for  every  task  in  the  con¬ 
tract.  These  task  earned  values  are  then  added  to  give  a 
measure  of  earned  value  for  the  entire  contract.  The 
earned  value  is  compared  with  the  actual  costs  incurred 
to  date  to  evaluate  contract  cost  performance.  Schedule 
status  is  provided  by  the  milestones  that  define  the 
tasks. 

A  tabular  example  of  how  this  process  works  is  shown 
on  the  next  page.  (The  milestones  and  cumulative  expen¬ 
ditures  are  not  given.) 
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Total 

Estimated  Cost 

$  250,000 

Total 

Costs  Incurred 

to  Date 

$  175,000 

Tasks 

Value  % 

Complete 

Earned  Value 

1 

$40,000 

100 

$  40,000 

2 

$60,000 

100 

60,000 

3 

$80,000 

70 

56,000 

4 

$40,000 

50 

20,000 

5 

$30,000 

10 

3,000 

$  179/000 


The  earned  value  is  $179,000,  which  indicates  that  a 
$4,000  underrun  on  efforts  has  been  incurred  to  date. 

The  TV  chart  shown  in  DID  V  in  Appendix  A  has  onlv 
one  column  for  the  earned  value  data.  This  column  is 
marked  "Percent  Complete."  When  the  contractor  submits 
his  original  plan,  he  will  place  the  dollar  value  of  each 
task  in  this  column.  The  periodic  reports  will  show  the 
percent  complete  of  each  task  in  this  column.  Only  one 
column  is  provided  on  the  TV  chart  for  the  following 
reason.  If  the  contractor  were  to  make  the  earned  value 
calculation,  he  could  be  motivated  to  make  the  earned 
value  more  optimistic  by  adjusting  his  percent  complete 
estimates.  This  possibility  can  be  minimized  if  the  con¬ 
tract  monitor  performs  these  calculations. 

A  worksheet  for  calculating  earned  value  over  the 
life  of  a  contract  is  shown  in  Figure  2.  The  contract 
monitor  lists  the  tasks  and  their  value  on  the  left-hand 
side  cf  the  worksheet.  As  contractor  reports  are  re¬ 
ceived,  the  monitor  lists  the  percent  complete  for  each 
task  and  calculates  its  earned  value.  The  total  contract 
earned  value  would  then  be  entered  at  the  bottom  of  the 
chart,  and  compared  to  the  actual  cost  to  that  date. 

Thus,  an  earned  value  trend  will  be  developed  over  the 
life  of  the  contract. 

Milestone  costing  provides  a  measure  of  earned  value 
by  associating  schedule  milestones  directly  with  the  pro¬ 
jection  of  cumulative  costs.  This  is  done  by  placing  the 
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effort's  major  milestones  directly  on  the  cumulative  cost 
curve,  as  illustrated  by  the  sample  report  for  DID  IV  in 
Appendix  A,  indicating  how  much  money  should  have  been 
expended  when  each  milestone  is  accomplished.  The  mile¬ 
stones  fit  on  the  curve  at  the  proper  dates  because  it  is 
assumed  that  the  cumulative  cost  curve  was  developed  by 
estimating  the  costs  for  each  task  and  assigning  those 
costs  to  the  time  period  over  which  the  task  is  scheduled. 
If  this  assumption  is  not  valid,  the  planning  function 
has  not  been  performed  properly  and  efforts  to  improve  it 
should  be  made. 

As  progress  is  made  on  the  effort,  the  contractor 
reports  expenditures  and  milestone  completions.  However, 
he  also  shows  actual  milestone  accomplishment  on  the 
cumulative  cost  curve,  as  illustrated  in  the  sample  re¬ 
port  for  DID  IV  in  Appendix  A.  When  one  milestone  is  ac¬ 
complished,  the  contractor  is  credited  with  an  earned 
value  equal  to  the  expected  cost  at  that  milestone.  This 
earned  value  is  then  compared  with  the  actual  cost  at 
that  milestone  to  measure  any  performance  variance. 

Milestone  costing  requires  somewhat  less  effort  by 
the  contractor  and  the  contract  monitor,  because  it  does 
not  require  estimates  of  percent  complete  and  calculation 
of  earned  value.  However,  it  also  has  a  limitation  which 
car  reduce  its  effectiveness.  Milestone  costing  does  not 
place  a  value  on  work,  or  tasks,  in  progress.  Since  many 
of  the  tasks  in  an  R&D  effort  are  usually  overlapping,  a 
number  of  tasks  will  be  partially  complete  when  any  given 
milestone  is  accomplished.  Efforts  on  these  tasks  could 
have  been  reduced  to  complete  the  present  milestone  on 
time  and  on  cost.  However,  the  projected  costs  at  the 
milestone  completion  include  some  costs  for  other,  as  yet 
incomplete,  tasks.  If  these  tasks  were  being  underspent, 
thus  disguising  an  overrun,  it  would  not  show  on  the  MC 
report.  Since  TV  does  place  a  value  on  tasks  in  progress, 
it  will  be  more  appropriate  for  contracts  that  have  sig¬ 
nificantly  concurrent  tasks. 

One  other  problem  can  limit  the  usefulness  of  the  MC 
and  TV  approaches,  or  any  other  approach  that  uses  a 
time-phased  plan,  to  measuring  earned  value.  This 
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situation  is  where  the  original  plans  are  "front-loaded"; 
i.e.,  this  means  that  tasks  scheduled  early  in  the  pro¬ 
gram  are  assigned  larger  than  realistic  "values"  and  the 
later  program  tasks  are  assigned  smaller  "values"  to  com¬ 
pensate.  Performance  early  in  the  contract  will  seem 
adequate  even  though  an  overrun  may  be  occurring.  Prob¬ 
lems  that  develop  will  show  up  in  the  later  stages  of  the 
report  where  actual  costs  are  compared  to  the  unrealisti¬ 
cally  low  estimates.  The  AFSPPF  contract  monitor  should 
carefully  review  and  evaluate  the  contractor's  original 
plans  to  ensure  that  the  cost  estimates  for  all  tasks  are 
not  front- loaded. 

A  set  of  "additive"  Data  Item  Descriptions  should 
be  used  to  implement  cost  and  schedule  reporting 
requirements  on  AFSPPF  contracts. 

Five  basic  Data  Item  Descriptions  have  been  recom¬ 
mended  for  AFSPPF' s  use  as  specific  means  of  implementing 
PMM&Co. 's  cost  and  schedule  recommendations.  The  data 
items  have  been  written  so  that  they  can  be  used  in  "ad¬ 
ditive"  fashion.  For  example,  the  R&D  Contract  Status 
Report/Narrative  could  be  used  by  itself  on  a  small  con¬ 
tract.  For  larger  contracts  that  required  more  sophisti¬ 
cated  reporting,  additional  data  items  would  ba  used,  and 
the  R&D  Contract  Status  Report/Narrative  would  become  the 
narrative  portion  of  the  entire  report. 

The  following  pages  contain  summary  descriptions  of 
each  of  the  five  DID's  with  guidelines  for  their  applica¬ 
tion  to  contracts. 

The  R&D  Contract  Status  Report/Narrative  can  be 
used  by  itself  or  as  the  textual  portion  of  more  sophis¬ 
ticated  reports.  It  specifies  the  content  and  organiza¬ 
tion  of  the  contractor's  report  with  the  following 
outline: 

I.  Highlights 

II.  Government  Actions  Required 

III.  Technical  Performance 

IV.  Schedule 
V.  Cost 
VI.  Contractual 
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The  Program  Schedule/Milestcne.  Accomplishment 
is  used  to  depict  the  program  schedule  and  report  on 
milestone  accomplishments.  No  cost  data  is  included  in 
this  report,  so  it  may  be  used  on  fixed-price  contracts. 
This  report  also  contains  the  basic  schedule  planning  and 
reporting  guidance  to  be  followed  when  submitting  MC  and 
TV  reports . 

The  Cumulative  Cost  Projection/Report  provides 
the  basic  means  of  reporting  expected,  and  actual  contract 
costs  as  a  function  of  time.  When  the  report  is  used  as 
a  plan  of  contract  costs,  detailed  cost  buildup  data  can 
be  required;  e.g.,  direct  labor  costs  per  task,  overhead 
and  general  and  administrative  rates,  materials  costs, 
etc.  The  contractor  is  required  to  submit  estimates  to 
complete  each  reporting  period  and  any  changes  that  have 
occurred  in  his  cost  buildup  data;  e.g.,  overhead  rates. 

The  Milestone  Costing  Plan/Report  represents 
the  means  of  levying  the  milestone  costing  approach  to 
measuring  earned  value  on  the  contractor.  It  references 
the  requirements  of  the  milestone  reporting  and  cumula¬ 
tive  cost  reporting  DID's  whose  requirements  should  also 
be  met  when  submitting  an  MC  report.  The  milestone  cost¬ 
ing  report  will  always  include  the  R&D  Contract  Status 
Report/Narrative  as  its  textual  portion. 

The  Task  Valuation  Plan/Report  represents  the 
means  of  levying  the  task  valuation  approach  to  measuring 
earned  value  on  the  contractor.  As  with  the  Milestone 
Costing  Plan/Report,  it  includes,  by  reference,  the  re¬ 
quirements  of  the  milestone  reporting  and  cumulative  cost 
reportingDID’s.  The  textual  portion  of  the  report  is 
provided  by  the  R&D  Contract  Status  Report/Narrative. 

The  applicability  matrix  in  Figure  3  shows  how  the 
different  reporting  requirements  should  be  applied  to 
various  contracts.  The  major  considerations  in  formulat¬ 
ing  the  matrix  are  highlighted  as  follows: 
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FIGURE  3 

APPLICABILITY  MATRIX 


2 


jAFSPPF  mission  j 

♦Specific  task  work  plans  should  be  submitted  for  AFSPPF  approval  before  work  begins  on 
individual  tasks- 


.  More  sophisticated  reporting  is  required 
for: 

.  a  large  contract? 

.  a  technically  complex  contract;  and 

.  a  contract  with  a  great  risk  of 
failure  to  the  facility's  mission. 

.  Fixed-price  contracts  require  no  cost 
reporting . 

.  The  MC  approach  tends  to  be  used  on  con¬ 
tracts  with  sequential  tasks,  where  the 
problem  of  valuing  work  in  process  will 
be  minimized. 

.  The  TV  approach  is  applicable  on  most 
cost-reimbursable  contracts  that  have  a 
number  of  definable  tasks  and  are  on  the 
order  of  magnitude  of  $30  thousand  or 
greater. 

.  On  large  service  contracts,  specific 
task  work  plans  should  be  submitted  for 
approval  before  work  begins  on  individual 
tasks , 

At  the  present  time,  the  use  of  CPM  scheduling  and 

reporting  of  more  than  one  level  of  detail  on 

AFSPPF  contracts  is  not  appropriate. 

AFSPPF  does  not  have  contracts  of  sufficient  size  to 
require  the  use  of  work  breakdown  structures  or  critical 
path  scheduling  data  for  reports.  However,  these  ap¬ 
proaches  are  potentially  useful  on  one  or  two  of  the 
major  hardware  procurements  planned  by  the  facility  in 
the  next  few  years.  If  these  procurements  do  material¬ 
ize,  use  of  CPM  by  the  contractor  should  be  considered  as 
a  basis  for  summary  schedule  reporting. 
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Implementation  of  the. five  recommended  DID's  re¬ 
quires  consideration  of  three  factors: 

.  the  requirements  of  AFSC/AFLC  Manual 
310-1; 

.  the  use  of  the  Contract  Data  Require¬ 
ments  List  (DD  Form  1423) ;  and 

.  the  potential  contractor  responses  to 
the  data  requirements. 

AFSC/AFLC  Manual  310-1  Requirements 

Since  the  five  DID's  discussed  are  not  included  in 
the  Air  Force's  Authorized  Data  List  (ADL),  which  is  part 
of  AFSC/AFLC  Manual  310-1,  they  will  have  to  be  approved 
before  they  can  be  used  as  AFSPPF  contracts.  AFSC/AFLC 
Manual  310-1  provides  for  the  use  of  the  following  three 
types  of  nonstandard  DID's: 

.  "U"  Data  Item  Description; 

.  Modified  Data  Item  Description;  and 

.  R&D  Data  Item  Description. 

These  are  described  in  Volume  I,  Pages  1  and  2,  of  the 
manual  as  follows: 

"(2)  "U"  Data  Item  Description.  A  unique  (non¬ 

standard)  DID  proposed  or  approved  for  use 
when  a  data  requirement  cannot  be  met  by  use 
or  modification  of  a  standard  Data  Item  De¬ 
scription.  A  "U"  data  item  description  is 
intended  for  limited  use  only  (for  example, 
one  time  only  or  until  such  time  as  a  stand¬ 
ard  equivalent  is  incorporated  in  the  ADL) . 

"(3)  Modified  Data  Item  Description.  A  standard 
or  approved  "U"  data  item  description  that 
meets  the  general  requirements  of  the  data 
needed  but  must  be  modified  to  comply  with 


specific  program  requirements.  This  modifi¬ 
cation  can  only: 

(a)  Reduce  the  scope  through  deletion  of 
words,  paragraphs,  or  sections. 

(b )  Clarify  usage  where  considered  necessary. 

(c)  Adjust  the  format  to  meet  program-peculiar 
requirements  within  the  content  and  scope 
of  the  original  data  item  (for  example, 
use  of  contractor  format) .  When  the 
format  is  mandatory,  the  change  must  be 
processed  as  a  "U"  DID. 

(d)  Cite  the  acceptability  of  similar  data 
which  will  satisfy  the  Air  Force  require¬ 
ment  which  may  have  been  prepared  under 
the  auspices  of  another  Government  agency 
(for  example,  FAA  or  Navy) . 

(e)  Provide  for  the  combination  of  two  related 
data  items  in  a  single  report. 

"(4)  R&D  Data  Item  Description.  A  DID  generated 
for  use  when  standard  Data  Item  Descriptions 
will  not  suffice  on  research,  exploratory  de¬ 
velopment,  advanced  development,  and  related 
studies  and  support  programs,  where  hardware, 
equipment,  reports,  or  other  data  delivered 
are  solely  for  feasibility  or  experimental  in¬ 
vestigations.  Within  this  context,  reprocure¬ 
ment  or  adaptation  for  operational  use  is  not 
envisioned  or  contemplated;  otherwise,  R&D 
Data  Item  Descriptions  are  to  be  treated  as 
"U"  data  items." 

Since  the  facility's  procurements  are  in  the  R&D 
area,  the  recommended  DID's  should  be  able  to  be  treated 
as  R&D  data  items.  R&D  data  items  can  be  approved  at  the 
"laboratory  level"  (see  AFSC/AFLC  Manual  310-1,  Volume  I, 
Page  5.2).  Thus,  approval  for  the  data  items  can  be  ob¬ 
tained  from  the  data  management  offices  at  the  agencies 
through  wnich  AFSPPF  personnel  process  their  procurement 
actions . 
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Two  of  the  recommended  items  (i.e.,  the  R&D  Contract 
Status  Report/Narrative  and  the  Program  Schedule/Mile¬ 
stone  Report)  are  similar  to  data  items  on  the  ADL  (i.e., 
the  R&D  Contract  Status  Report,  A-113,  and  the  Program 
Schedule  Report,  M-103) .  Since  the  recommended  data 
items  explain,  in  detail,  the  content  of  the  approved 
data  items  as  opposed  to  adding  to  their  content,  they 
could  be  considered  as  modified  data  items  and  not  re¬ 
quire  approval. 

Use  of  the  Contract  Data  Requirements  List 
(DD  Form  1423) 


The  actual  levying  of  a  reporting  requirement  on  a 
contractor  is  accomplished  through  the  use  of  DD 
Form  1423.  This  form,  which  is  attached  to  and  is  part 
of  the  contract,  lists  the  reports  to  be  submitted  by  the 
contractor,  the  timing  of  submission,  number  of  copies, 
and  other  pertinent  data.  It  is  important  that  DD 
Form  1423  be  prepared  properly  so  that  the  contractor 
will  submit  reports  as  they  are  required  by  the 
government. 

Two  elements  of  the  information  on  DD  Form  1423  that 
are  particularly  important  are  the  timing  of  report  sub¬ 
missions  and  the  relation  of  the  submission  of  a  data 
item  plan  to  a  data  item  report  against  that  plan.  In 
most  cases,  the  data  items  recommended  should  be  sub¬ 
mitted  monthly.  However,  monthly  reports  are  often  re¬ 
ceived  three  and  four  weeks  after  the  close  of  the  month 
to  which  they  pertain.  Such  a  delay  can  reduce  the  value 
of  the  information  submitted  because  it  is  too  late  to 
respond  to  the  situation  described.  Management  action 
contemplated  as  a  result  of  the  late  report  is  often  in¬ 
effectual  because  the  situation  has  usually  changed  sig¬ 
nificantly.  The  DD  Form  1423  should  require  that  peri¬ 
odic  reports  be  received  (or  postmarked)  by  the  tenth  day 
of  the  month  following  the  month  covered  in  the  report. 

The  data  items  recommended  in  this  report  are  writ¬ 
ten  so  that  they  will  serve  as  a  requirement  for  a  plan 
and  reports  against  that  plan.  This  stipulation  should 
be  made  clear  in  the  "remarks"  section  of  DD  Form  1423. 


-25- 


The  plan  should  be  submitted  by  the  contractor  early  in 
the  effort,  approximately  10  days  after  the  contract 
start  date.  The  periodic  reports  should  then  be  sub¬ 
mitted  monthly  against  that  plan.  The  first  report  would 
cover  the  first  month  of  contract  effort  and  thus  be  sub¬ 
mitted  about  40  days  after  the  plan.  If  desirable,  these 
approximate  times  can  be  varied  to  meet  contractor  ac¬ 
counting  periods. 

Potential  Contractor  Responses  to  Data  Requirements 

When  required  to  submit  detailed  reports,  contrac¬ 
tors  often  place  very  high  costs  on  the  data  in  the  hope 
that  the  government  will  remove  the  requirement.  AFSPPF 
personnel  must  be  aware  of  this  possibility  and  have  some 
feel  for  the  cost  of  a  data  requirement  to  the  contractor. 

None  of  the  DID's  recommended  in  this  report  should 
cost  the  contractor  a  great  deal.  Milestone  costing  re¬ 
quires  only  that  the  contractor  relate  cumulative  costs 
directly  to  milestones,  both  of  which  are  common  data 
requirements.  The  relation  between  the  two  should  come 
directly  from  the  contractor's  plan. 

Task  valuation  requires  the  contractor  to  report  his 
expected  costs  for  each  contract  task;  again,  this  data 
should  be  avail  \ble  in  the  contractor's  plan.  In  addi¬ 
tion,  an  estimate  of  the  percent  complete  of  each  task  is 
required  for  eac  1  report.  This  estimate  is  to  be  made  by 
the  person  in  tne  contractor's  organization  directly  re¬ 
sponsible  for  accomplishing  the  work,  and  is  not  based  on 
the  costs  that  the  contractor  is  expending  on  this  task. 
Hence,  TV  does  not  place  any  requirements  on  the  contrac¬ 
tor  for  accumulating  costs  for  each  task. 

Any  data  requirements  levied  by  the  government  and 
their  costs  become  the  subject  of  negotiation  between  the 
government  and  the  contractor.  Initial  contractor  pro¬ 
posals  or  costs  need  not  be  taken  as  absolute.  In  par¬ 
ticular,  care  should  be  taken  to  closely  examine  contrac¬ 
tors  who  place  a  high  cost  on  these  data  items  because 
they  may  have  something  to  hide  or  may  have  improperly 
planned  their  efforts.  Since  much  of  the  data  required 
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is  from  plans,  the  lack  of  proper  planning  may  be  the 
cause  of  the  high  costs.  Improper  planning  is  a  major 
cause  of  contract  performance  deficiencies  and  should  be 
eliminated  if  uncovered  during  contract  negotiations. 
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V.  TECHNICAL  PERFORMANCE  REPORTING  REQUIREMENTS 


This  section,  addressed' to  Paragraph  4.2  of  the  con-  f 

tract  statement  of  work  entitled,  "Develop  Technical  Per-  j 

formance  Reporting  Requirements,"  presents  recommendations  j 

for  the  development,  by  AFSPPF  contractors,  of  technical 
achievement  plans  and  technical  performance  reporting 

against  these  plans.  f 


Statement  of  the  Problem 

The  recommendations  presented  in  Section  IV  for  im¬ 
proved  cost  and  schedule  reporting  are  directed,  in  part, 
at  obtaining  a  better  evaluation  of  program  progress  and 
eliminating  surprise  overruns.  It  has  been  PMM&Co.'s  ex¬ 
perience  that  although  improved  cost  and  schedule  reporting 
substantially  improves  the  ability  to  detect  problems  and 
take  corrective  management  action  surprise  overruns  are 
still  not  eliminated.  Eliminating  surprise  overruns  re¬ 
quires  a  significantly  better  appraisal  of  the  technical 
performance  status  of  the  project  than  the  government  cus¬ 
tomer  has  typically  had  in  the  past. 

To  a  large  extent,  technical  performance  measurement 
is  performed  through  personal  contact  between  government 
and  contractor  technical  personnel.  This  is  true  both  with 
AFSPPF  and  other  government  agencies  on  both  large  and  small 
projects.  Nonetheless,  the  effectiveness  of  technical  per¬ 
formance  measurement  through  personal  contact  is  limited  by 
the  monitor's  workload,  the  technical  complexity  of  the 
efforts  involved,  and  the  geographic  dispersion  of  the 
facility's  contractors.  A  contract  monitor,  responsible 
for  several  development  projects,  may  not  have  time  for  as 
much  personal  contact  with  each  contractor  as  he  would  desire. 

It  is  desirable  that  the  contract  monitors  obtain, 
from  the  contractor's  report  data,  the  maximum  visibility 
into  the  technical  status  of  the  project.  Hence,  systematic 
technical  performance  reporting  in  addition  to  the  C03t  and 
schedule  reporting,  is  a  major  contractor  reporting 
requirement. 
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Objective 

At  this  time,  AFSPPF  has  a  technical  performance  meas¬ 
urement  system  in  which  AFSPPF  personnel  personally  evaluate 
the  contractor's  technical  performance.  AFSPPF  personnel, 
and  particularly  the  contract  monitors,  use  "eyeball-to- 
eyeball"  contacts,  design  reviews,  evaluation  of  test  re¬ 
sults,  and  other  means  to  monitor  the  technical  status  of 
the  project.  However,  this  form  of  technical  performance 
measurement  is  not  systematized,  is  not  part  of  the  overall 
project  planning  and  control  system,  and  may  or  may  not  in¬ 
clude  specific  evaluation  of  technical  progress  early  in  the 
project.  PMM&Co.'s  objective  was  to  identify  and  recommend 
a  means  of  obtaining  systematic  technical  performance  re¬ 
porting  that  would  enable  the  monitors  to  identify  problems 
early  in  the  project. 


Approach 

The  overall  PMM&Co.  technical  approach  was  to  develop 
plans  of  measurable  technical  achievement  and  specify  re¬ 
porting  of  progress  against  these  plans.  The  Technical 
Achievement  Plan  (TAP)  would  encompass  the  following 
characteristics : 

.  parameters  to  be  tracked  and  required 
specification  values; 

.  development  of  a  plan  against  which  prog¬ 
ress  would  subsequently  be  evaluated; 

.  identification  of  critical  parameters  to 
be  tracked  and  their  specified  values; 

.  identification,  within  the  TAP,  of  interim 
events  at  which  technical  performance 
could  be  measured. 

.  identification  of  technical  results  ex¬ 
pected  at  these  interim  technical  achieve¬ 
ment  events; 

.  maximum  use  of  tests  and  other  objective 
measurements  (e.g.,  design  calculations) 
as  technical  achievement  events  (TAE) ;  and 

.  identification  of  sufficient  TAE's  early 
in  the  project. 
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After  such  plans  have  been  developed,  a  technical  report 
would  be  generated  as  each  TAE  occurs.  The  report  would 
identify  and  evaluate  variances  from  planned  results. 

As  a  result  of  the  P:..M&Co.  examination  of  the  AFSPPF 
contract  files  and  discussions  with  project  managers,  a 
format  for  a  TAP  that  encompasses  the  above  characteristics 
was  derived.  The  technical  performance  recommendations 
described  below  are  based  on  this  TAP  format. 


Recommendations 


The  technical  performance  portion  of  the  PMM&Co.  study 
resulted  in  four  recommendations.  These  recommendations 
are  as  follows : 

.  develop  TAP's  for  hardware  development 
projects ; 

.  require  technical  performance  reporting 
against  the  TAP's; 

.  require  integrated  variance  analysis;  and 

.  use  a  modified  form  of  the  TAP  for  study 
contracts . 

The  following  pages  contain  detailed  discussions  of 
PMM&Co.'s  technical  performance  recommendations. 

AFSPPF  contractors  should  be  required  to  use  TAP's  on 

major  hardware  development  projects. 

The  TAP  will  identify  the  technical  parameters  of  the 
equipment  or  system  and  their  specified  value,  the  interim 
achievement  points  (i.e.,  TAE's)  at  which  progress  toward 
achieving  the  specified  parameter  values  can  be  measured, 
and  the  value  that  is  expected  to  be  measured  at  each  TAE. 

A  sample  TAP  is  presented  as  Figure  4. 

The  first  column  of  the  TAP  lists  the  parameters  to 
be  measured,  and  the  second  column  lists  the  specified 
value  of  those  parameters.  For  instance,  the  specification 
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TECHNICAL  ACHIEVEMENT  EVENTS 


FIGURE  4  -  TECHNICAL  ACHIEVEMENT  PlAN 


for  the  printing  rate  is  some  number  of  prints  per  second. 
This  specification  value  should  be  more  precisely  described 
in  backup  documentation.  In  this  instance,  there  may  be 
more  than  one  mode  of  operation  with  different  printing 
rates.  Thus,  the  specification  value  has  to  be  stated  in 
terms  of  certain  conditions  and,  in  many  cases,  certain 
tolerances. 

The  remaining  columns  (with  the  exception  of  the  com¬ 
ments  column)  show  the  TAE's.  At  these  events,  eight  of 
which  have  been  identified  in  the  figure,  progress  toward 
some  or  all  of  the  listed  parameters  can  be  measured.  A 
checkmark  indicates  that  the  parameter  will  be  measured  at 
that  TAE.  For  example,  at  TAE  1,  Precise  Feed  Servo  Bread¬ 
board  Test  Complete,  the  printing  rate,  length  of  print, 
and  print  spacing  can  be  measured.  Furthermore,  a  check 
indicates  that  the  value  expected  at  that  TAE  is  the  same 
as  the  specif ication  value.  If,  for  various  reasons,  some¬ 
thing  different  than  the  specification  value  is  expected 
(possibly  as  a  result  of  having  a  breadboard  and  not  the 
final  equipment),  then  the  expected  value  would  be  indicated. 

A  further  reason  exists  for  having  something  other 
than  the  checkmark  in  one  of  the  boxes.  This  is  the  situa¬ 
tion  where  the  test  is  for  something  other  than  the  speci¬ 
fication  value  itself.  For  instance,  at  TAE  2,  Reel  Servo 
Breadboard  Test  Complete,  transport  rate  and  stability  will 
be  tested  rather  than  the  printing  rate.  Detailed  explana¬ 
tions  of  these  events  should  be  described  in  backup 
documentation. 

Diagonal  lines  are  drawn  in  each  box  where  technical 
performance  measurements  will  be  made  so  that  the  planned 
value  can  be  put  in  the  upper  left  corner  and,  when  reports 
are  submitted,  the  achieved  value  can  be  entered  in  the 
lower  right  corner.  In  this  way,  the  TAP  becomes  a  dual- 
purpose  document  that  can  be  used  for-  both  technical  achieve¬ 
ment  planning  and  reporting. 

Techn i cal  performance  reporting  against  the  t echnical 

achievement  plans  should  be  required. 

Once  the  TAP's  described  above  have  been  developed  by 
the  contractor  and  approved  by  the  government,  the  contractor 
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will  report  technical  performance  achievement  against  the 
plan.  Two  kinds  of  reporting  will  be  required.  First,  a 
report  will  be  required  whenever  a  TAE  occurs,  and,  second, 
periodic  reports  will  be  submitted  with  periodic  cost  and 
schedule  reports. 

The  reports  submitted  when  a  TAE  occurs  will  include 
the  TAP  and  an  accompanying  narrative  report.  The  TAP  will 
show  in  the  lower  right  corner  of  the  appropriate  boxes  the 
value  achieved  for  each  test.  Since  this  number  by  itself 
may  be  of  limited  value,  the  contractor  should  also  submit 
accompanying  narrative  documentation  that  expresses  more 
fully  the  test  results.  This  documentation  will  include 
what  was  achieved,  the  confidence  in  the  numbers  that  were 
achieved,  and  the  implications  of  the  test  results.  Where 
variances  exist,  their  cost,  schedule,  and  technical  impact 
will  be  discussed.  Corrective  management  action  being  taken 
and  any  management  action  the  contractor  feels  the  govern¬ 
ment  should  take  should  be  included. 

For  the  periodic  reports,  the  technical  performance 
status  can  be  included  in  the  R&D  Contract  Status  Report/ 
Narrative  (see  DIB  I,  Appendix  A)  and  should  be  accompanied 
by  the  latest  edition  of  the  TAP. 

Contractors  should  be  required  to  perform  an  integrated 

variance  analysis. 

The  full  effectiveness  of  the  TAP,  and  subsequent 
reporting,  is  not  realized  unless  the  contract  monitor  is 
able  to  analyze  his  cost,  schedule,  and  technical  perform¬ 
ance  reports  together  to  identify  root  problems. 

Technical  Achievement  Plans  should  be  used  on  study 

contracts  with  some  modification. 


The  TAP's  described  above  usually  would  not  be  appro¬ 
priate  to  a  study  contract,  because  the  contractor  is  not 
required  to  produce  a  product  with  specific  performance 
requirements.  The  study  contract  usually  consists  of  a 
number  of  subobjectives  that  do  have  a  specified  nu¬ 
merical  value.  For  example,  a  typical  study  may  be  broken 
down  into  the  following  subobjectives: 
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.  define  duplicate  quality; 

.  develop  objective  means  of  measuring 
quality ; 

.  develop  criteria  to  rank  the  quality  of 
samples; 

.  determine  statistically  the  best  objective 
quality  indicator. 

In  the  TAP  for  a  study,  a  list  of  subojectives  may  be 
more  specifically  defined  than  is  done  in  this  example. 

TAE's  that  are  appropriate  to  these  objectives  can  then  be 
identified.  Typically,  these  would  not  be  test  events. 

More  likely,  they  would  be  the  major  milestones  of  the  study 
that  would  be  included  in  the  milestone  report.  The  TAP 
then  could  designate  the  milestones  at  which  formal  proj¬ 
ect  reviews  would  be  held.  The  contract  monitor  would  iden¬ 
tify  what  information  should  be  made  available  for  these 
reviews,  the  technical  aspects  of  effort  to  be  considered, 
and  what  results  are  expected  from  the  reviews. 

The  TAE's  on  a  hardware  contract  were  defined  to  be 
objective  treasures  of  technical  achievement.  However,  it 
is  usually  not  possible  to  develop  such  measures  for  study 
contracts.  Hence,  the  TAE's  for  a  study  contract  will  gen¬ 
erally  be  subjective  rather  than  objective  measures.  Where 
objective  TAE's  on  a  study  contract  can  be  defined,  they 
should  be  used.  For  these  study  programs,  technical  per¬ 
formance  reporting  would  take  place,  as  for  the  hardware 
programs,  when  the  TAE  occurred.  Overall  technical  status 
to  date  would  be  incorporated  with  the  periodic  cost  and 
schedule  narrative  reports.. 


Implementation 


To  a  large  extent,  the  exact  nature  of  the  TAP  and  the 
steps  in  its  implementation  are  a  function  of  the  particular 
project.  The  longer,  larger  (in  cost) ,  and  more  complex 
the  development  program,  the  more  appropriate  a  TAP.  How¬ 
ever,  the  TAP  can  be  used  with  hardware  developments  of 
almost  any  size.  For  small  programs,  the  plan  might  not 
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include  much  more  than  a  1isting  of  the  contract  specifica¬ 
tions,  their  values,  and  one  or  two  interim  points  at  which 
these  can  be  measured.  In  general,  some  form  of  a  TAP  can 
be  used  for  any  hardware  development  effort  in  which  the 
contractor  is  charged  with  developing  equipment  to  meet 
contract  specifications. 

Modification  of  the  TAP's,  as  discussed  in  the  last 
recommendation  in  this  section,  can  be  used  to  formalize 
the  planning  and  control  of  study  contract  work  efforts. 

Such  TAP's  should  usually  be  of  assistance  on  study  contracts 
that  cost  $50  thousand  or  more  and  that  last  at  least  three 
months . 

Technical  Achievement  Plans  and  reporting  are  not  felt 
to  be  appropriate  for  services  contracts  and  for  the  pur¬ 
chase  of  off-the-shelf  equipment.  Additionally,  in  working 
with  the  contract  monitors  to  test  the  validity  of  a  TAP  for 
a  software  development,  it  could  not  be  verified  that  tech¬ 
nical  achievement  plans  would  be  useful.  For  the  particular 
contract  studied,  there  was  no  equivalent  to  the  test  events 
that  would  serve  as  an  interim  measure  of  technical  perform¬ 
ance,  and  there  were  no  detailed  specifications  to  describe 
the  project. 

Implementation  of  technical  achievement  planning  and 
reporting  has  several  steps  chat  are  common  to  both  devel¬ 
opment  and  study  contracts.  These  are  the  following: 

.  Inclusion  of  the  technical  achievement  plan¬ 
ning  and  reporting  requirements  in  the  Re¬ 
quest  for  Proposal,  or  its  equivalent. 

.  Description  by  the  contractor,  in  his  pro¬ 
posal,  of  how  he  will  specifically  meet  the 
government's  planning  and  reporting 
requirement. 

.  Submission  by  the  contractor,  shortly  after 
contract  award,  of  a  Technical  Achievement 
Plan. 

.  Approval  by  AFSPPF  of  that  plan. 
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.  Submission  by  the  contractor  of  subsequent 
reports  against  the  plan.  (The  plan  would 
be  changed  only  when  there  was  a  correspond¬ 
ing  contract  change.) 

With  this  mode  of  operation,  the  contractor  is  allowed 
to  develop  his  own  plan,  but  the  government  has  the  right 
of  approval.  Thus,  the  monitor  has  an  opportunity  to  ensure 
that  the  contractor  has  met  the  government's  requirement. 

Since  TAP's  and  reports  are  data  requirements,  they 
must  be  included  on  the  "Contract  Data  Requirements  List," 

DD  Form  1423.  This  can  be  done  by  using  data  item  S-117, 
"Technical  Reports,1'  from  the  ADL  in  AFSC/AFLC  Manual  310-1. 
The  specific  requirements  for  the  plans  and  reports  would 
be  included  in  the  statement  of  work  as  referenced  on  the 
DD  Form  1423. 

The  recommendations  made  for  technical  performance 
planning  and  reporting  should  place  no  significant  burden 
on  AFSPPF  contractors.  They  may  require  a  contractor  to 
reformat  his  plans  and  deliver  them  to  the  government,  which 
he  has  not  been  required  to  do  in  the  past.  Nonetheless, 
the  requirements  involve  nothing  over  and  above  effective 
engineering  management.  Thus,  the  contractor  should  not 
have  significant  difficulty  in  implementing  the  require¬ 
ment,  nor  should  it  cost  him  a  significant  amount  of  money 
to  implement  such  a  system. 


VI.  MANAGEMENT  INFORMATION  SYSTEM  RECOMMENDATIONS 


This  section,  addressed  to  Paragraph  4.3  of  the  con¬ 
tract  statement  of  work  entitled  "Develop  Management  Infor 
mation  System,"  presents  recommendations  for  improving  the 
use  of  contractor  reporting  data  within  AFSPPF. 


Statement  of  Problem 


AFSPPF 's  research  and  development  activities  involve 
the  management  of  a  number  of  relatively  small  contracts. 

These  contracts  involve  a  number  of  technical  disciplines, 
contractors,  procuring  agencies,  etc.  It  is  sometimes 
difficult  for  facility  personnel  to  keep  apprised  of  the 
status  of  each  of  the  efforts  in  which  they  are  interested. 

While  this  problem  exists  for  those  individuals  inter¬ 
ested  i.i  only  a  few  of  the  contracts,  the  problem  is  magni¬ 
fied  for  those  persons  involved  in  the  entire  R&D  activi¬ 
ties  of  AFSPPF.  In  particular,  the  Commander  and  his  staff, 
the  Director  of  Research  and  Development,  and  the  Management 
Operations  Officer  are  required  to  be  familiar  with  all  of 
the  technical  facility's  efforts.  This  familiarity  is  re¬ 
quired  because  of  the  frequent  command  decisions  necessary 
concerning  R&D  matters,  and  the  continual  interfacing  of 
the  individuals  mentioned  with  other  related  organizations. 

A  large  amount  of  contractual  data  is  available  at  AFSPPF 
because  it  is  required  by  contract  monitors  to  effectively 
manage  their  efforts.  Because  so  much  data  is  available, 
no  one  person  in  the  facility  can  have  a  detailed  knowledge 
of  all  the  data  relevant  to  all  of  the  contracts.  Hence, 
contract  data  must  be  made  available  in  summary  form  to 
support  R&D  management  decision-making  at  AFSPPF. 

It  is  also  important  to  note  that  AFSPPF 's  primary  re¬ 
search  and  development  mission  is  not  only  to  effectively 
manage  R&D  contracts.  The  facility  has  a  broader  mission 
in  that  it  must  accomplish  certain  technical  objectives 
(i.e.  ,  design  and  development  of  a  state-of-the-art  hardware 
device)  in  support  of  an  operational  requirement.  The  con¬ 
tractual  effort  is  only  one  aspect  of  achieving  the  objective, 
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although  a  major  one.  The  facility  must  also  identify  and 
define  a  requirement,  develop  ways  to  meet  the  requirement, 
initiate  a  procurement  action,  manage  a  contract,  carry  out 
test  and  evaluation  programs,  and  oversee  the  installation 
of  the  equipment.  Hence,  decision-making  in  the  R&D  area 
must  always  be  performed  with  the  technical  objective  in 
mind,  in  addition  to  the  specifics  of  any  particular  con¬ 
tract.  Contractor  performance  data  must  be  used  so  that 
it  supports  the.  accomplishment  of  the  technical  objective, 
in  addition  to  the  management  of  the  contract. 


Approach 


PMM&Co.'s  original  approach  to  improving  the  communi¬ 
cation  of  contractor  performance  data  within  AFSPPF  was  to 
create  a  management  information  center  (MIC) .  A  MIC  would 
consist  of  a  number  of  wall  charts  containing  contractor 
performance  data,  in  summary  form,  and  any  other  administra¬ 
tive  data  relevant  to  contract  performance  (e.g.,  funding). 
Thus,  all  relevant  contract  data  would  be  available  in 
quickly  understood  format.  This  set  of  charts  would  serve 
as  the  principal  means  of  communicating  contractor  perform¬ 
ance  data  to  the  Commander  and  would  provide  a  framework 
for  making  contract-related  decisions. 

PMM&Co.'s  studies  have  indicated  that  this  approach 
is  not  appropriate  for  AFSPPF  use  due  to  the  small  size  of 
the  contracts  involved  and  the  active  role  taken  by  AFSPPF 
command  personnel  in  the  R&D  area.  In  particular,  the 
facility  Commander  frequently  interfaces  with  higher  au¬ 
thority  and  other  organizations  on  R&D  matters.  This  inter¬ 
facing  requires  him  to  have,  readily  at  hand,  detailed  infor¬ 
mation  on  the  status  of  many.  R&D  projects.  Hence,  an  MIC 
could  only  contain  summary  data  and  would  not  be  transport¬ 
able.  The  Commander  would  probably  be  familiar  with  more 
detailed  data  on  a  number  of  the  projects  than  was  available 
in  the  MIC.  Further,  an  MIC  that  consists  of  charts  would 
require  significant  effort  to  create  and  maintain,  thus 
potentially  reducing  the  frequency  of  updating  which,  in 
turn,  would  reduce  the  timeliness  of  the  data  presented. 

Since  the  MIC  approach  was  not  felt  to  be  appropriate, 
PMM&Co.  broadened  the  scope  of  its  inquiries  in  the  management 
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information  systems  area.  In  particular,  PMM&Co.  looked 
at  how  facility  personnel  use  contractor  performance  data 
in  contract  management,  and  how  this  data  is  used  in  the 
accomplishment  of  technical  objectives  at  AFSPPF. 


Objective 

In  light  of  the  above  findings,  the  objective  of 
PMM&Co.' s  task  was  to  make  practical  recommendations  to 
improve  the  way  AFSPPF  uses  contractor  performance  data 
in  R&D  management.  In  particular,  these  recommendations 
should  assist  the  contract  monitor  in  his  efforts  to  ef¬ 
fectively  manage  an  R&D  contract;  should  improve  the  way 
in  which  data  is  made  available  to  AFSPPF  command  personnel 
to  support  their  management  requirements;  and  should  empha¬ 
size  the  facility's  broad  R&D  mission  of  accomplishing  a 
technical  objective  in  response  to  an  operational  requirement. 


Recommendations 


Four  recommendations  in  the  management  information 
systems  area  are  discussed  below.  These  relate  to  the  man¬ 
agement  role  of  the  contract  monitors,  use  of  work  plans 
by  the  monitors,  an  alternative  to  an  MIC,  and  planning  of 
a  number  of  technically  related  contracts. 

The  contract  monitors'  full  responsibility  for 
meeting  the  cost,  schedule,  and  technical  objectives 
of  a  contractual  effort  should  be  reaffirmed. 


AFSPPF  contract  monitors  have  a  wide  range  of  authority 
and  responsibility  in  regard  to  contractual  matters.  At 
times,  considering  the  involvement  of  command  personnel  and 
Air  Force  procurement  agencies  in  AFSPPF  R&D  activities, 
the  extent  of  these  responsibilities  is  not  entirely  clear. 
While  the  contract  monitor  does  not  have  the  authority  to 
make  the  major  decisions  relating  to  a  contract  (such  as 
whether  to  accept  a  proposal  or  terminate  an  effort)  ,  he  does 
have  responsibility  for  the  overall  management  of  the  effort. 
Further,  this  responsibility  includes  all  aspects  (i.e., 
cost,  schedule,  and  technical)  of  contractual  performance. 
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6 

AFSPPF  command  personnel  should  reaffirm  that  it  is 
the  monitor's  responsibility  to  take  the  primary  role  in  1 

the  management  of  an  effort,  whether  this  involves  pre-  i 

ser.ting  alternatives,  with  recommendations  to  the  Commander 
for  a  decision,  or  taking  initiative  on  his  own  part.  In 
fact,  recommendations  concerning  contractor  performance 
data  reporting  have  been  made  to.  support  the  management 
activities  of  the  contract  monitor.  If  the  monitor  does 
not  understand  or  accept  the  full  range  of  his  responsi¬ 
bilities,  effective  contract  management  at  AFSPPF  will  not 
be  possible. 

The  responsibilities  of  the  contract  monitor  are,  as 
has  been  stated,  managerial  in  nature.  Hence,  the  term 
"contract  monitor"  is  somewhat  of  a  misnomer,  and  would 
appropriately  be  changed.  The  term  "project  manager"  or 
"project  engineer"  would  be  more  descriptive  of  the  moni¬ 
tor's  responsibilities. 

The  use  of  work  plans  by  contract  monitors  on  con¬ 
tracts  or  technical  efforts  should  be  required. 

As  described  above,  the  contract  monitor  has  cost, 
schedule,  and  technical  responsibility  for  the  contractual 
effort.  However,  the  facility's  R&D  mission  has  been  more 
broadly  defined  to  include  the  accomplishment  of  a  technical 
objective  in  response  to  an  operational  requirement.  This 
includes  a  number  of  activities,  such  as  managing  a  pro¬ 
curement  action  or  carrying  out  a  test  and  evaluation  pro¬ 
gram,  in  addition  to  the  specific  contract  activities. 

Since  the  monitor  also  has  responsibility  for  these  broader 
technical  objectives,  he  should  have  plans  showing  how  the 
technical  objective  is  to  be  met.  These  "work  plans"  would 
serve  the  following  functions : 

.  provide  a  baseline  against  which  to  measure 
accomplishment  of  the  technical  objective 
that  will  meet  the  operational  requirement; 

.  reemphasize  the  monitor's  responsibility  for 
meeting  the  technical  objective,  in  addition 
to  his  cost,  schedule,  and  technical  respon¬ 
sibility  for  the  specific  contractual  effort; 
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.  provide  a  framework  for  making  contractual 
decisions,  such  as  extensions,  scope  changes, 
etc.; 

.  provide  a  formalized  means  of  communication 
between  contract  monitors  and  AFSPPF  command 
personnel. 

The  content  of  the  work  plans  will  vary  depending  upon 
the  nature  of  the  effort  involved.  Typically,  they  would 
consist  of  two  to  four  pages  of  written  material  plus  ex¬ 
hibits.  The  work  plans  would  include  the  following  data: 

.  the  requirement  that  the  contract  effort  is  to 
meet  (i.e.,  a  state-ient  of  the  problem); 

.  the  technical  objectives  or  the  performance 
specifications  needed  to  meet  the 
requirement; 

.  schedules  (both  contractor  and  AFSPPF 
activities) ; 

.  cost  (expected  and  acceptable;  i.e.,  if 
the  contractor's  proposal  was  for  twice 
as  much  money  as  was  expected,  would  it 
still  be  accepted) ; 

.  high  risk  areas  (technical,  cost,  or 
schedule) ; 

.,  management  approach  (including  data  require¬ 
ments,  review  meetings,  and  support  require¬ 
ments  from  other  AFSPPF  personnel) ; 

•  the  decisions  required  by  the  Commander,  with 
dates . 

Work  plans  should  be  written  by  the  monitors  and  up¬ 
dated  periodically,  probably  quarterly,  for  most  contracts. 
The  work  plans  would  be  reviewed  and  approved  by  the  Direc¬ 
tor  of  Research  and  the  facility  Commander.  Since  the  plans 
should  assist  contract  monitors  and  supplement  interpersonal 
management,  they  will  not  be  standardized.  They  should  be 
adjusted  to  meet  the  particular  requirements  of  the  effort 
in  question. 
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In  addition  to  the  use  of  work  plans,  one  other  device 
may  be  of  assistance  to  the  contract  monitors.  This  is 
the  periodic  report  review  checklist  that  is  shown  as  Figure 
5.  This  checklist  would  serve  as  a  guideline  to  the  monitor 
when  reviewing  contractor  reports.  In  addition,  it  would 
provide  a  convenient  way  to  record  his  comments  on  and  ac¬ 
tion  taken  in  response  to  the  reports.  The  annotated  check¬ 
list  could  be  circulated  with  the  contractor  report  so  that 
other  facility  personnel  will  see  the  report  and  the  moni¬ 
tor's  comments  together. 

A  Management  Information  Notebook  should  be  used  as 

a  practical  alternative  to  a  management  information 

center. 

Because  of  its  inflexibility  and  updating  workload,  an 
MIC  was  not  recommended  as  a  means  of  communicating  con¬ 
tractual  data  to  the  AFSPPF  Commander.  A  Management  Infor¬ 
mation  Notebook  (MIN) ,  with  two  pages  of  data  on  each  con¬ 
tract,  would  be  a  practical  alternative  to  an  MIC.  Formats 
for  the  data  are  shown  in  Figures  6s  and  6b. 

The  MIN  would  contain  summary  data  on  the  facility's 
R&D  contracts ,  with  emphasis  being  placed  on  the  technical 
objective  of  the  effort.  This  data  will  provide  the  Com¬ 
mander  with  a  historical  framework  within  which  to  evaluate 
program  status  and  make  program  decisions.  In  particular, 
the  charts  would  contain  the  following  data: 

.  identifying  information; 

.  description  of  the  effort  and  requirement 
to  be  met; 

.  schedule  data,  including  the  date  the  fa¬ 
cility  must  have  the  output  of  the  effort; 

.  technical  risk  areas; 

.  cost  sensitivity  (e.g.,  would  the  equipment 
still  be  procured  if  a  singificant  over¬ 
run  occurred?) ; 

.  summary  of  review  meetings; 
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Figure  5 


SAMPLE  REPORT  REVIEW  CHECKLIST 

REVIEW  SHEET:  PERIODIC  CONTRACT  STATUS  REPORT 

I.  Report  Covers  (date) : 

When  Rec'd:  Timely?: 

II.  Report  Meet  DD  1423/1664  Req'mts 


III.  Variances/Problems  (identify  problems,  impact,  and  action  required): 

A.  Technical: 

B.  Cost: 

C.  Schedule: 

IV.  Government  Actions  Required: 

V.  Estimates  To  Complete  (monitor's  estimate  of  completion  date, 

cost,  and  performance;  does  this  vary  from  contractor's  estimate; 
is  this  acceptable?) : 

VI'.  Overall  Evaluation  of  Contractor  Performance: 


r 
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FIGURE'  6A  MANAGEMENT  INFORMATION  NOTEBOOK  FORMAT 


$162,000 


MANAGEMENT  INFORMATION  NOTEBOOK  FORMAT 


.  financial  data,  including  projected  and 
actual  costs  and  earned  value; 

.  status  information  fay  month  (the  status 
of  a  number  of  contract  parameters  can  be 
tracked  monthly  with  red,  yellow,  cr  green 
pencil;  blank  spaces  are  provided  below 
the  para.aeters  shown  so  that  any  other 
items  of  importance  to  a  particular  con¬ 
tract  can.  be  inserted) . 

Since  there  would  only  be  two  pages  of  data  per  contract, 
the  HIN  would  be  small  and  relatively  easy  to  keep  current, 
'thus,  when  the  facility  Commander  travels,  he  could  take  the 
notebook  with  him. 

Hie  contract  monitor  would  have  the  responsibility  for 
filling  out  and  updating  the  charts  for  his  projects. 

This  should  not  be  burdensome,  since  most  of  the  data  on 
the  charts  will  cone  from  the  monitor's  work  plans.  The 
Management  Operations  Officer  should  have  general  responsi¬ 
bility  for  maintaining  the  MIS. 

The  MIS  should  supplement  the  facility's  present 
'.eekly  R&D  briefings,  as  opposed  to  replacing  them.  The 
focus  of  the  weekly  briefings  is  on  updating  recent  changes 
in  contract  status  and  providing  for  information  transfer 
on  R*D  natters  among  facility  personnel-  The  MIS,  on  the 
other  hand,  has  a  longer  terra  orientation  because  its  focus 
is  on  the  technical  objectives  of  the  effort,  which  places 
contract  performance  in  a  historical  context - 

When  a  number  of  --.F5P?F  contracts  lead  to  the  sane 
technical  objective,  planning  should  be  performed  on 
a  technical  area  basis. 


Host  AFSPPF  R&D  contracts  are  somewhat  independent 
in  that  problems  with  one  do  not  directly  affect  the  per¬ 
formance  of  others.  Presently,  however,  AFSPPF  is  engaged 
in  efforts  in  one  "technical  area"  that  includes  the  efforts 
cf  a  number  of  contractors,  both  on  fixed-price  and  cost- 
reimbursable  efforts.  These  efforts  require  significant 
interfacing  between  the  contractors  and  between  contractor 
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and  AFSPPF  personnel.  When  such  a  situation  exists,  it 
is  important  that  the  planning  and  managing  of  the  efforts 
involved  are  done  as  a  whole,  since  problems  with  one  effort 
could  affect  the  others. 

Efforts  recently  made  to  perform  such  technical  area 
planning  have  had  positive  results.  If  other  areas  of  tech¬ 
nical  effort  requiring  a  number  of  contractual  actions  de¬ 
velop,  planning  should  be  performed  on  the  area  basis. 

This  planning  should  include  required  AFSPPF  activities  in 
addition  to  contractor  tasks,  and  the  interrelationships 
between  AFSPPF  and  contractor  tasks. 
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VII.  SUPPLEMENTARY  RECOMMENDATIONS 


In  the  course  of  FKM&Co. 1  s  effort  on  this  contract, 
three  recommendations  were  developed  which  concerned  the 
relationship  between  AFSPPF  and  its  contractors.  These 
recommendations,  which  are  supplementary  to  the  contrac- 
tuar  statement  of  work,  are  summarized  below. 


1.  The  facility  should  avoid  concurrent  cost- reimbursable 
and  fixed-price  contracts  v/ith  one  contractor  in  tech¬ 
nically  related  areas. 

If  a  contractor  has  two  contracts  with  AFSPPF  at  the 
same  time,  one  fixed  price  and  one  cost  reimbursab.1  e,  he 
may  be  tempted  to  charge  some  of  the  costs  of  the  fixed- 
price  effort  to  the  cost-reimbursable  effort.  Whenever 
possible,  the  facility  should  avoid  this  situation  by 
either  delaying  the  efforts  or  making  the  two  efforts  one 
contract.  When  this  is  not  possible,  the  contractor’s 
charges  to  the  cost- reimbursable  contract  must  be  closely 
reviewed  to  ensure  that  they  belong  to  that  effort. 


2.  "he  facility  should  insist  that  all  cost- reimbursable 
contracts  have  limitations  on  the  overhead  and  gen¬ 
eral  arc:  adnir.is-rative  rates. 

AFSPPF  contractors  have,  in  a  number  of  instances, 
experienced  large  increases  in  overhead  (0/H)  and  general 
and  administrative  (G&A)  rates.  These  increases  have 
caused  significant  overruns  on  facility  contracts  because 
0/H  and  G&A  typically  represent  50  percent  of  the  total 
contract  cost.  For  example,  the  contractor's  cost  esti¬ 
mates  on  a  recently  initiated  program  were  as  follows 
(figures  are  approximate) : 
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Direct  Labor 

$ 

37,000 

0/H  (81%) 

30,000 

Direct  Costs 

Consultants 

1,600 

Travel 

1,200 

Materials 

3,400 

_ 

6,200 

Total  Research  Costs 

73,200 

G&A  (25%) 

— 

18,300 

91,500 

- 

Fee 

— 

7,500 

TOTAL  COST  PLUS  FIXED 

i  FEE 

$_ 

99,000 

In  this  example,  0/H  and  G&A  accounted  for  $48,300,  which 
is  approximately  50  percent  of  the  total  cost  plus  fixed 
fee.  If  the  0/H  and  G&A  rates  go  up  10  percent  on  this 
contract,  ar.  overrun  of  approximately  5  percent  will  be 
incurred  (since  0/H  and  G&A  are  one-half  of  the  total 
costs) .  If  the  0/H  and  G&A  rates  are  higher,  the  poten¬ 
tial  overrun  can  be  much  greater. 

This  problem  can  be  avoided  by  placing  "limitation 
of  overhead  rate”  clauses  in  the  facility's  contracts. 

In  many  instances,  contractors  will  probably  not  want  to 
accept,  such  clauses.  However,  contractors  that  have  that 
attitude  are  probably  anticipating  rate  increases,  making 
tha  clauses  all  the  more  desirable  for  AFSPPF. 


3.  AFSPPF  personnel  should  take  an  active  role  in  con¬ 
tract  negotiations . 

Many  contractual  matters  (e.g.,  cost  rates,  data  re¬ 
quirements,  and  personnel  to  be  used  by  the  contractor) 
are  decided  daring  contract  negotiations.  At  this  time, 
the  government  has  the  greatest  leverage  in  determining 
the  nature  of  the  effort.  Since  AFSPPF  personnel  are 
mora  familiar  with  the  facility's  requirements  than 


i 
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personnel  in  the  procuring  agencies,  they  should  work 
closely  with  the  procurement  personnel  to  ensure  that 
the  facility's  requirements  are  met.  Facility  personnel 
should  also  understand  that  all  contractual  matters  are 
the  subject  of  negotiations  before  the  contract  is 
signed  and,  at  that  time,  should  take  full  advantage  of 
their  prerogatives  to  gain  f-vorable  contract  terms. 
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Appendix  A 


DATA  ITEM  DESCRIPTIONS 


This  appendix  contains  five  data  item  descriptions 
(DID)  to  be  used  in  the  implementation  of  the  recommenda 
tions  made  in  this  report.  These  DID's  are  as  follows: 

I.  R&D  Contract  Status  Report/Narrative 

II.  Program  Schedule/Milestone  Accomplishment 

III.  Cumulative  Cost  Projection/Report 

IV.  Milestone  Costing  Plan/Report 

V.  Task  Valuation  Plan/Report 


I.  R&D  CONTRACT  STATUS  REPORT/NARRATIVE 

A.  DESCRIPTION/PURPOSE 

The  R&D  Contract  Status  Report/Narrative  is  designed 
as  a  periodic,  recurring  report  to  keep  management 
informed  of:  (1)  the  progress  made  on  R&D  contracts  during 
the  previous  reporting  period;  (2)  any  problems  encountered 
which  will  nave  an  impact  on  the  cost,  schedule,  or  techni¬ 
cal  objectives  of  the  effort;  and  (3)  any  requirements  for 
action  by  the  government. 

B.  APPLICATION/INTERRELATIONSHIP 

The  report  will  be  used  on  cost-reimbursable  contracts 
in  the  R&D  area.  It  may  be  used  on  fixed-price  contracts 
with  the  deletion  of  cost  data.  When  more  sophisticated 
reporting  is  required,  the  narrative  report  will  setve  as 
the  textual  portion  of  the  entire  report.  The  narrative 
section  will  introduce  the  other  data  items  and  provide 
any  detailed  explanations  required. 

C.  PREPARATION  INSTRUCTIONS 

1.  Format 

The  report  will  be  written  according  to  the  following 
outline,  with  additional  sections  added  as  required: 

I.  HIGHLIGHTS 

II.  GOVERNMENT  ACTIONS  REQUIRED 

III.  TECHNICAL  PERFORMANCE 

IV.  SCHEDULE 

V.  COST 

VI.  CONTRACTUAL 

The  reports  should  be  brief,  but  not  incomplete.  Where 
clarity  will  not  be  lost,  outline  format  will  be  acceptable. 
If  it  is  desirable  to  include  a  quantity  of  data  in  one 
area,  exhibits  should  be  used. 

2 .  Content 

The  report  sections  outlined  above  should  contain,  as 
appropriate,  the  following  information: 
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HIGHLIGHTS  -  All  reports  longer  than  one 
or  two  typewritten  pages  will  have  a  high¬ 
lights  section.  This  section  will  be  one- 
half  page  or  less  and  will  outline  the 
major  points  to  be  made  in  the  report. 

GOVERNMENT  ACTIONS  REQUIRED  -  Any  govern¬ 
ment  actions  required  in  the  next  reporting 
period,  or  overdue  from  past  reporting  periods, 
or  new  requirements  for  future  periods  will 
be  listed. 

TECHNICAL  PERFORMANCE  -  The  following  will 
be  included: 

.  description  of  approach  and  progress 
during  the  reporting  period,  supported 
by  reasons  for  any  change  in  approach 
reported  previously,  including  a  dis¬ 
cussion  on  the  effect  of  any  changes 
on  the  accomplishment  of  the  contrac¬ 
tual  objectives; 

.  description  of  any  major  items  of  ex¬ 
perimental  or  special  equipment  pur¬ 
chased  or  constructed  during  the  re¬ 
porting  period. 

*  .  notification  of  any  changes  in  key 

personnel  associated  with  the  contract 
during  the  reporting  period; 

.  summary  of  substantive  information 

derived  from  noteworthy  trips,  meetings, 
and  special  conferences  held  in  connec¬ 
tion  with  the  contract  during  the  re¬ 
porting  period; 

.  other  accomplishments;  i.e.,  scientific 
papers  published,  etc. 

SCHEDULE  -  The  following  will  be  involved: 

.  statement  of  schedule  (milestone) 
achievements ; 


-53- 


.  explanation  of  any  rescheduling  of 
contractual  efforts  (original  sched¬ 
ule  will  always  be  shown) ; 

.  estimate  of  the  completion  date  of 
the  contract  and  the  next  significant 
milestone. 

.  COST  -  The  following  will  be  involved: 

.  statement  of  funds  expended  to  date; 

.  statement  relative  to  the  apparent 
sufficiency  of  contract  funds  to 
achieve  the  objective  of  the  contract; 

.  explanation  of  any  variances  between 
planned  and  actual  expenditures; 

.  statement  of  any  variances  in  cost 
buildup  data  from  plan  (including 
direct  labor  rates,  overhead  rates, 
general  and  administrative  rates, 
labor  hours,  materials  costs,  etc.). 

.  CONTRACTUAL  -  All  contractual  matters  (such 
as  submission  of  CCN's,  etc.)  that  arise 
during  the  contract  should  be  detailed  in 
this  section.  These  items  may  be  the  sub¬ 
ject  of  other  correspondence,  but  they 
should  also  be  included  in  this  section. 
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II.  PROGRAM  SCHEDJLE/MILESTONE  ACCOMPLISHMENT 

A.  DESCRIPTION/PURPOSE 

The  Program  Schedule/Milestone  data  item  portrays: 

(1)  the  phases,  major  milestones,  and  tasks  of  the  effort 
required  to  meet  the  contractual  objectives;  and  (2)  the 
significant  program  progress  made  by  the  contractor  be¬ 
tween  reporting  dates. 

B .  APPLICATION/INTERRELATIONSHIP 

This  data  item  is  applicable  to  most  RSD  efforts, 
including  hardware  development,  software  development,  and 
studies.  It  can  also  be  used  on  fixed-price  contracts  if 
schedule  data  is  required  by  the  government  for  effective 
management  of  the  program.  This  report  should  always  be 
accompanied  by  DID  I,  R&D  Contract  Status  Report/Narra¬ 
tive,  which  will  provide  a  narrative  explanation  of  pro¬ 
gram  plans  and  progress.  When  more  sophisticated  report¬ 
ing,  which  includes  program  schedules  and  milestone  ac¬ 
complishments,  is  required  (see  DID  IV,  Milestone  Costing, 
and  DID  V,  Task  Valuation)  the  guidance  in  DID  II  should 
be  followed. 

C.  PREPARATION  INSTRUCTIONS 

1.  The  contractor  will  prepare  Milestone  Re¬ 
ports  using  AFSC  Form  103  (see  Exhibit  1) ,  showing  sched¬ 
ules  expressed  in  terms  of  (a)  phases,  which  cover  spans 
of  time,  and  (b)  milestones,  which  are  actions  or  accom¬ 
plishments  that  can  be  scheduled  for  a  specific  point  in 
time  and  evaluated  on  the  basis  of  actual  accomplishment. 
These  schedules  will  be  shown  as  realistically  as  can  be 
estimated  at  the  time  of  preparation. 

2.  This  data  item  may  be  prepared  in  the  con¬ 
tractor's  own  format,  if  it  is  equivalent  to  AFSC 

Form  103.  In  any  case,  the  charts  should  be  clear  and 
have  a  scale  as  large  as  the  normal  form  size  will  per¬ 
mit.  As  the  technical  effort  progresses,  if  the  sched¬ 
ules  are  changed,  the  original  schedules  will  always  be 
shown . 
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3.  Slippages,  or  other  discrepancies  between 
planned  and  actual  accomplishment,  and  reasons  for 
changes  in  forecasts  will  be  supported  in  the  accompany¬ 
ing  data  item,  R&D  Contract  Status  Report/Narrative. 
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III.  CUMULATIVE  COST  PROJECTION/REPORT 


A.  DESCRIPTION/PURPOSE 

The  objective  of  the  Cumulative  Cost  Projection/Re¬ 
port  is  to:  (1)  provide  a  projection  of  contract  costs, 
graphed  cumulatively  over  time:  and  (2)  provide  for  the 
periodic  reporting  of  the  actual  costs  incurred  on  the 
contract  for  comparison  with  the  projection.  Provision 
is  also  made  in  the  data  item  format  for  inclusion,  in 
tabular  form,  of  more  detailed  contract  performance  data. 

B.  APPLICATION/INTERRELATIONSHIP 

This  data  item  can  be  used  on  a  wide  range  of  con¬ 
tracts  in  the  R&D  area,  except  for  firm-fixed-price  con¬ 
tracts.  It  will  often  be  used  in  conjunction  with  DID  II, 
Program  Schedule/Milestone  Accomplishment,  to  provide 
cost  data  for  program  evaluation. 

C.  PREPARATION  INSTRUCTIONS 

1.  At  the  beginning  of  the  contractual  effort, 
the  contractor  will  prepare  a  projection  of  cumulative 
costs  to  be  incurred  during  the  contract  using  the  Cumu¬ 
lative  Cost  Form,  presented  as  Exhibit  2,  or  the  contrac¬ 
tor's  equivalent.  The  rows  for  additional  data  will  be 
used  if  it  were  instructed  to  do  so  on  DD  Form  1423, 
"Contract  Data  Requirements  List." 

2.  The  contractor  w^ll  submit  periodic  updates 
of  the  Cumulative  Cost  Form  showing  actual  contract 
costs.  The  frequency  and  dates  of  submission  of  these 
updates  will  be  given  on  DD  Form  1423.  These  updated  re¬ 
ports  will  always  include  an  estimate  to  complete  of 

the  actual  expenditures  curve.  This  estimate  to  complete 
will  be  the  contractor's  latest  estimate  of  the  cost  to 
complete  the  work  not  yet  done,  the  completion  date,  and 
how  the  costs  will  be  expended  as  a  function  of  time. 

3.  Total  costs,  unless  otherwise  indicated  on 
DD  Form  1423,  will  be  defined  as  total  contract  costs, 
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excluding  fixed  fee.  Total  costs  will  include  direct 
labor,  overhead,  materials,  subcontract  costs,  travel, 
etc. 


4.  The  contractor  will  not  change  the  original 
projected  cost  curve  unless  specific  permission  has  been 
granted  by  the  responsible  Air  Force  authority.  The  es¬ 
timate  to  complete,  however,  will  only  be  shown  for  the 
latest  period. 

5.  When  this  data  item  is  used  as  a  plan  of 
contract  costs  over  time,  in  addition  to  a  report,  more 
detailed  cost  buildup  may  be  required.  The  cost  of  each 
contract  task  will  be  given  and  broken  down  into  direct 
labor,  overhead,  materials,  travel,  and  general  and  ad¬ 
ministrative  categories.  The  rates  and  assumptions  used 
in  making  the  cost  buildups  will  be  clearly  identified. 


IV.  MILESTONE  COSTING  PLAN/REPORT 


A.  DESCRIPTION/PURPOSE 

Milestone  Costing  is  a  method  of  directly  associating 
planned  expenditures  of  contract  funds  with  the  scheduled 
completion  of  contract  milestones.  Comparison  of  the  ac¬ 
tual  and  planned  costs  when  a  milestone  is  completed  will 
provide  a  measure  of  the  earned  value  of  the  contractual 
effort. 


B .  APPLICATION/INTERRELATIONSHIP 

This  data  item  will  be  applicable  to  R&D  contracts 
which  have  a  number  of  intermediate  milestones  and  tasks 
which  do  not  greatly  overlap.  It  should  not  be  used  on 
contracts  which  require  more  sophisticated  reporting,  such 
as  DID  V,  Task  Valuation.  Such  a  contract  would  be  one  in 
which  the  individual  tasks  were  concurrent  in  time,  since 
a  number  of  tasks  would  be  in  process  at  each  milestone. 

Since  Task  Valuation  estimates  the  value  of  work  in  proc¬ 
ess,  whereas  Milestone  Costing  does  so  less  precisely. 

Task  Valuation  would  be  appropriate  for  these  contracts. 

C.  PREPARATION  INSTRUCTIONS 

1.  The  contractor  will  prepare  a  Program  Schedule 
and  Cumulative  Cost  Projection  as  outlined  in  DID's  II  and 
III.  These  two  plans  should  not  be  constructed  independently. 
The  Cumulative  Cost  Projection  should  be  developed  by  sum¬ 
ming  the  expected  costs  of  each  of  the  tasks  in  the  Program 
Schedule.  When  the  Cumulative  Cost  Projection  has  been 
completed,  the  milestones  from  the  Program  Schedule  should 
be  placed  on  the  cost  curve  (see  Exhibit  3) .  The  milestones 
should  be  numbered  for  easy  reference  to  the  Program 
Schedule . 


2.  As  effort  on  the  contract  progresses,  actual 
expenditures  are  graphed,  and  actual  completion  of  mile¬ 
stones  are  shown  on  the  actual  cumulative  expenditure  curve. 
Each  time  a  report  is  submitted,  a  projection  to  completion 
of  both  milestones  and  costs  should  be  included  (see  Ex¬ 
hibit  3) .  The  Program  Schedule/Milestone  Report  should 
also  be  updated  each  reporting  period. 
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3.  The  guidance  given  for  preparing  and  submitting 
the  Program  Schedule/Milestone  Report  and  the  Cumulative 
Cost  Projection  Report,  DID's  II  and  III,  should  be  followed 
except  when  it  conflicts  with  the  above  instructions. 

4.  Contractor  equivalent  forms  may  be  used  to 
increase  clarity  of  the  reports  or  to  reduce  preparation 
costs . 
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EXHIBIT  3-  MILESTONE  COSTING  EXAMPLE 


V.  TASK  VALUATION  PLAN/REPORT 


A.  DESCPIPTION/PURPOSE 

The  Task  Valuation  Plan/Report  is  a  method  of  devel¬ 
oping  a  relationship  between  the  cost  and  work  performance 
p’ ans  of  a  contract  so  that  the  expected  cost  of  each  con¬ 
tract  task/activity  is  identified.  Progress  reporting 
against  this  plan  is  then  accomplished  by  reporting  mile¬ 
stone  accomplishments,  costs  incurred,  and  the  percent  com¬ 
plete  for  each  of  the  contract  tasks/activities. 

B.  APPLICATION/INTERRELATIONSHIP 

This  data  item  will  be  applicable  to  R&D  contracts 
which  consist  of  a  number  of  identifiable  tasks.  It  should 
also  be  useful  on  study  and  software  contracts  if  there  are 
enough  tasks  in  the  contract,  the  tasks  are  not  greatly 
overlapping  in  time,  and  the  milestones  defining  the  start 
and  finish  of  the  tasks  are  well  defined. 

This  data  item  description  relates  milestone  planning/ 
reporting  to  cumulative  cost  projection/reporting  and  allo¬ 
cates  the  total  contract  value  to  the  individual  tasks. 
Hence,  the  requirements  of  milestone  reporting  (DID  II)  and 
cumulative  expenditure  reporting  (DID  III)  apply  to  the 
Task  Valuation  Plan/Report  except  where  specific  changes 
are  noted. 

C.  PREPARATION  INSTRUCTIONS 

1.  The  contractor  will  prepare  a  Milestone  Sched¬ 
ule  as  he  would  if  complying  with  DID  II.  However,  he  will 
use  the  format  shown  in  Exhibit  4  which  has  the  additional 
column  "%  complete."  When  submitting  the  TV  report  as  a 
plan  at  the  start  of  the  contractual  effort,  the  projected 
cost  of  performing  each  task  shown  will  be  entered  in  this 
column.  When  reports  are  submitted,  the  contractor  will 
evaluate  the  percent  complete  for  each  of  the  tasks  and 
enter  the  values  in  the  column.  (See  note  for  an  explana¬ 
tion  of  percent  complete.) 
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2.  The  contractor  will  also  prepare  an  estimated 
cumulative  cost  curve  (DI.  Ill)  at  the  beginning  of  an 
effort.  The  projection  will  be  updated  with  actual  cost 
data  and  submitted  with  the  milestone  schedule  shown  in 
Exhibit  3. 


3.  It  is  important  that  the  "as  of  date"  be  both 
the  date  to  which  the  estimates  of  percent  complete  apply, 
and  the  date  through  which  expenditures  are  reported. 


NOTE:  Estimates  of  the  percent  complete  of  each  task 

should  be  made  by  the  individual  directly  respon¬ 
sible  for  the  performance  of  the  effort.  The  esti¬ 
mate  should  be  based  on  the  resources  (man-hours , 
materials,  etc.)  which  have  been  expended  on  the 
effort  and  that  are  estimated  to  be  required  to 
finish  the  task.  For  instance,  if  an  engineer 
estimates  that  50  hours  have  been  expended  on  a 
design  task,  and  50  additional  hours  will  be  re¬ 
quired,  the  task  is  50  percent  complete,  even  if 
the  original  estimate  for  the  task  was  75  hours. 

The  estimates  should  be  based  on  the  efforts  re¬ 
quired  to  complete  the  tasks,  and  not  on  the  origi¬ 
nal  estimated  cost  of  the  effort. 
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